Amendments to the Claims: 



1 . (Currently Amended) A computer-implemented method of searching 
hypotheses for locations of objects in a playback imago corresponding to a recorded 
image generated by a graphical user interface (GUI) of an application program 
comprising: 

capturing the a selected playback imag e, the playback image being one of a series 
of display images generated over time by a graphical user inteface (GUI) of an 
application program being tested by a cognitive control framework (CCF) system 
in a playback phase in response to simulated user inputs to the application 
program ; 

detecting at least one active display object in the arecorded image , the recorded 
image being one of a series of display images generated over time by the GUI of the 
application program being tested and captured during a recording phase of the CCF 
system based on user inputs to the application program, the user inputs, time interval 
between user inputs, and resulting display data forming an execution scenario script for 
use in testing the GUI of the application program during the playback phase for 
generating playback images, the at least one active display object being a portion of the 
recorded image being acted upon as a result of user input ; 

searching subsets of hypotheses of locations of display objects in ftem the 
captured playback image for an a matching display object corresponding to the detected 
at least one active display object of the recorded image according to predetermined 
criteria; 

recalculating old actions for the matching display object in the playback image by 
applying actions according to an the execution scenario script and loading a next set of 
data, when the corresponding matching display object is found; and 

checking dynamic conditions , including the time between user inputs . 

2. (Currently amended) The method of claim 1 , further comprising repeating the 
capturing, detecting, searching, recalculating, and checking for each of a series of both of 
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the playback and recorded images according to the execution scenario script to 
automatically test the GUI of the application program . 

3. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a first search using the predetermined criteria of content and layou t of display 
objects in the GUI of the application program under test . 

4. (Currently Amended) The method of claim I, wherein searching comprises 
performing a first search using the predetermined criteria of size and distance of display 
objects in the GUI of the application program under test . 

5. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a second search using the predetermined criteria of content and sizes of 
obj ects in the GUI of the application program under test . 

6. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a second search using the predetermined criteria of layout and distances in the 
GUI of the application program under test . 

7. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a third search using the predetermined criteria of content and sizes of 
additional objects, and layout of display objects in the GUI of the application program 
under test . 

8. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a third search using the predetermined criteria of sizes of active objects and 
distances in the GUI of the application program under test . 

9. (Currently Amended) The method of claim 1, wherein searching comprises 
performing a third search using the predetermined criteria of content for active objects in 
the GUI of the application program under test . 
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10. (Currently Amended) An article comprising: a machine accessible readable 
medium containing instructions, which when executed, result in searching hypotheses for 
locations of objects in a playback imago corresponding to a recorded imago generated by 
a graphical user interface (GUI) of an application program by 

capturing the playback image; 

dctocting at loast ono activo objoct in tho recorded imago; 

searching subsets of hypothosos from tho playback imago for an object according 

to predetermined criteria; 

recalculating old actions for the object in the playback image by applying actions 

according to an execution scenario and loading a next set of data, when the object is 

found; and 

checking dynamic conditions- 
capturing a selected playback image, the playback image being one of a series of 
display images generated over time by a graphical user inteface (GUI) of an 
application program being tested by a cognitive control framework (CCF) system 
in a playback phase in response to simulated user inputs to the application 
program; 

detecting at least one active display object in a recorded image, the recorded 
image being one of a series of display images generated over time by the GUI of the 
application program being tested and captured during a recording phase of the CCF 
system based on user inputs to the application program, the user inputs, time interval 
between user inputs, and resulting display data forming an execution scenario script for 
use in testing the GUI of the application program during the playback phase for 
generating playback images, the at least one active display object being a portion of the 
recorded image being acted upon as a result of user input; 

searching subsets of hypotheses of locations of display objects in the captured 
playback image for a matching display object corresponding to the detected at least one 
active display object of the recorded image according to predetermined criteria; 
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recalculating old actions for the matching display object in the playback image by 
applying actions according to the execution scenario script and loading a next set of data, 
when the corresponding matching display object is found; and 

checking dynamic conditions, including the time between user inputs. 

1 1 . (Currently Amended) The article of claim 1 0, wherein instructions to search 
comprise instructions to perform a first search using the predetermined criteria of content 
and layout of display objects in the GUI of the application program under test . 

12. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a first search using the predetermined criteria of size 
and distance of display objects in the GUI of the application program under test . 

13. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a second search using the predetermined criteria of 
content and sizes of objects of display objects in the GUI of the application program 
under test . 

14. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a second search using the predetermined criteria of 
layout and distances of display objects in the GUI of the application program under test . 

15. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a third search using the predetermined criteria of content 
and sizes of additional objects, and layou t of display objects in the GUI of the application 
program under test . 

16. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a third search using the predetermined criteria of sizes 
of active objects and distances of display objects in the GUI of the application program 
under test. 
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17. (Currently Amended) The article of claim 10, wherein instructions to search 
comprise instructions to perform a third search using the predetermined criteria of content 
for active objects of display objects in the GUI of the application program under test . 

18. (Currently Amended) A cognitive control framework (CCF) system for 
automatically controlling execution of an application program under test having a 
graphical user interface comprising: 

a recording component adapted to capture user input data and images displayed 
by the GUI graphical user interface during a recording phase of execution of the 
application progra m being tested , and to analyze the captured user input data and 
displayed images to generate an execution scenario script during the recording phase, and 

a playback component adapted to perform image analysis on images displayed by 
the GUI graphical user interface as a result of processing the simulated user input data 
during the aplayback phase and captured displayed images from the recording phase, the 
playback component being adapted to search hypotheses for locations of objects in a 
playback image corresponding to a recorded image by 
capturing the playback imag e ; 

detecting at least one active object in the recorded image; 

searching subsets of hypotheses from the playback image for an object 
according to predetermined criteria; 

recalculating old actions for the object in the playback image by applying 
actions according to an execution scenario and loading a next set of data, when 
the object is found; and 

chocking dynamic conditions. 
capturing a selected playback image, the playback image being one of a series of 
display images generated over time by the GUI of the application program being 
tested by the CCF system in the playback phase in response to simulated user 
inputs to the application program; 

detecting at least one active display object in the recorded image, the recorded 
image being one of a series of display images generated over time by the GUI of the 
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application program being tested and captured during a recording phase of the CCF 
system based on user inputs to the application program, the at least one active display 
object being a portion of the recorded image being acted upon as a result of user input; 

searching subsets of hypotheses of locations of display objects in the captured 
playback image for a matching display object corresponding to the detected at least one 
active display object of the recorded image according to predetermined criteria; 

recalculating old actions for the matching display object in the playback image by 
applying actions according to the execution scenario script and loading a next set of data, 
when the corresponding matching display object is found; and 

checking dynamic conditions, including the time between user inputs. 

19. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a first search using the predetermined criteria of content and layout of display 
objects in the GUI of the application program under test . 

20. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a first search using the predetermined criteria of size and distance of display 
objects in the GUI of the application program under test . 

21. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a second search using the predetermined criteria of content and sizes of 
obj ects of display objects in the GUI of the application program under test . 

22. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a second search using the predetermined criteria of layout and distances of 
display objects in the GUI of the application program under test . 

23. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a third search using the predetermined criteria of content and sizes of 
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additional objects, and layout of display objects in the GUI of the application program 
under test . 

24. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a third search using the predetermined criteria of sizes of active objects and 
distances of display objects in the GUI of the application program under test . 

25. (Currently Amended) The system of claim 18, wherein searching comprises 
performing a third search using the predetermined criteria of content for active objects of display 
objects in the GUI of the application program under test . 
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